The present disclosure relates generally to managing resource consumption in a computing system, and more specifically to a resource control system for controlling resource consumption in a computing system.
Database management systems typically allow users to define their own code, for example user-defined functions (UDFs), which are run under the control of the database management system. These UDFs may run ‘fenced’, meaning that they are not executed within the same process as the main database engine, they rather may be run in a separate external process spawned by the database management system, and the database management system optionally provides the UDF with input parameters, such as input data tuples if the UDF operates on cells stored within the database, and optionally receives output, such as output data tuples generated from the UDF. Since the user can provide the UDF as binary compiled code, the database management system typically has no information about runtime resource requirements of the UDF. However, it is desirable for the database management system to be able to monitor and react to resource consumption by the UDFs.
One solution is to impose a hard memory limit, e.g., operating system data ‘ulimit’, on external processes. However, this has the drawback that it is applied to each UDF process individually. When the memory limit is reached, this approach may result in receiving out-of memory error messages for future memory allocations for fenced user code, which will typically cause the related process to terminate. Further, the database management system may wish to spawn multiple separate user-defined processes, and thus there is no control over the aggregated memory consumption from all user-defined functions. Another solution is to instrument the target program code of the UDF such that additional program code is injected in a way that any memory allocations by the user-supplied code are intercepted. This can be cumbersome for programmers.